Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning

Structured pruning and quantization are promising approaches for reducing the inference time and memory footprint of neural networks. However, most existing methods require the original training dataset to fine-tune the model. This not only brings heavy resource consumption but also is not possible for applications with sensitive or proprietary data due to privacy and security concerns. Therefore, a few data-free methods are proposed to address this problem, but they perform data-free pruning and quantization separately, which does not explore the complementarity of pruning and quantization. In this paper, we propose a novel framework named Unified Data-Free Compression(UDFC), which performs pruning and quantization simultaneously without any data and fine-tuning process. Specifically, UDFC starts with the assumption that the partial information of a damaged(e.g., pruned or quantized) channel can be preserved by a linear combination of other channels, and then derives the reconstruction form from the assumption to restore the information loss due to compression. Finally, we formulate the reconstruction error between the original network and its compressed network, and theoretically deduce the closed-form solution. We evaluate the UDFC on the large-scale image classification task and obtain significant improvements over various network architectures and compression methods. For example, we achieve a 20.54% accuracy improvement on ImageNet dataset compared to SOTA method with 30% pruning ratio and 6-bit quantization on ResNet-34.Comment: ICCV202

SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration

The study of sparsity in Convolutional Neural Networks (CNNs) has become widespread to compress and accelerate models in environments with limited resources. By constraining N consecutive weights along the output channel to be group-wise non-zero, the recent network with 1$\times$N sparsity has received tremendous popularity for its three outstanding advantages: 1) A large amount of storage space saving by a \emph{Block Sparse Row} matrix. 2) Excellent performance at a high sparsity. 3) Significant speedups on CPUs with Advanced Vector Extensions. Recent work requires selecting and fine-tuning 1$\times$N sparse weights based on dense pre-trained weights, leading to the problems such as expensive training cost and memory access, sub-optimal model quality, as well as unbalanced workload across threads (different sparsity across output channels). To overcome them, this paper proposes a novel \emph{\textbf{S}oft \textbf{U}niform \textbf{B}lock \textbf{P}runing} (SUBP) approach to train a uniform 1$\times$N sparse structured network from scratch. Specifically, our approach tends to repeatedly allow pruned blocks to regrow to the network based on block angular redundancy and importance sampling in a uniform manner throughout the training process. It not only makes the model less dependent on pre-training, reduces the model redundancy and the risk of pruning the important blocks permanently but also achieves balanced workload. Empirically, on ImageNet, comprehensive experiments across various CNN architectures show that our SUBP consistently outperforms existing 1$\times$N and structured sparsity methods based on pre-trained models or training from scratch. Source codes and models are available at \url{https://github.com/JingyangXiang/SUBP}.Comment: 14 pages, 4 figures, Accepted by 37th Conference on Neural Information Processing Systems (NeurIPS 2023

Learning Global-aware Kernel for Image Harmonization

Image harmonization aims to solve the visual inconsistency problem in composited images by adaptively adjusting the foreground pixels with the background as references. Existing methods employ local color transformation or region matching between foreground and background, which neglects powerful proximity prior and independently distinguishes fore-/back-ground as a whole part for harmonization. As a result, they still show a limited performance across varied foreground objects and scenes. To address this issue, we propose a novel Global-aware Kernel Network (GKNet) to harmonize local regions with comprehensive consideration of long-distance background references. Specifically, GKNet includes two parts, \ie, harmony kernel prediction and harmony kernel modulation branches. The former includes a Long-distance Reference Extractor (LRE) to obtain long-distance context and Kernel Prediction Blocks (KPB) to predict multi-level harmony kernels by fusing global information with local features. To achieve this goal, a novel Selective Correlation Fusion (SCF) module is proposed to better select relevant long-distance background references for local harmonization. The latter employs the predicted kernels to harmonize foreground regions with both local and global awareness. Abundant experiments demonstrate the superiority of our method for image harmonization over state-of-the-art methods, \eg, achieving 39.53dB PSNR that surpasses the best counterpart by +0.78dB $\uparrow$; decreasing fMSE/MSE by 11.5\%$\downarrow$/6.7\%$\downarrow$ compared with the SoTA method. Code will be available at \href{https://github.com/XintianShen/GKNet}{here}.Comment: 10 pages, 10 figure

mTOR: A Potential New Target in Nonalcoholic Fatty Liver Disease

The global prevalence of nonalcoholic fatty liver disease (NAFLD) continues to rise, yet effective treatments are lacking due to the complex pathogenesis of this disease. Although recent research has provided evidence for the “multiple strikes” theory, the classic “two strikes” theory has not been overturned. Therefore, there is a crucial need to identify multiple targets in NAFLD pathogenesis for the development of diagnostic markers and targeted therapeutics. Since its discovery, the mechanistic target of rapamycin (mTOR) has been recognized as the central node of a network that regulates cell growth and development and is closely related to liver lipid metabolism and other processes. This paper will explore the mechanisms by which mTOR regulates lipid metabolism (SREBPs), insulin resistance (Foxo1, Lipin1), oxidative stress (PIG3, p53, JNK), intestinal microbiota (TLRs), autophagy, inflammation, genetic polymorphisms, and epigenetics in NAFLD. The specific influence of mTOR on NAFLD was hypothesized to be divided into micro regulation (the mechanism of mTOR’s influence on NAFLD factors) and macro mediation (the relationship between various influencing factors) to summarize the influence of mTOR on the developmental process of NAFLD, and prove the importance of mTOR as an influencing factor of NAFLD regarding multiple aspects. The effects of crosstalk between mTOR and its upstream regulators, Notch, Hedgehog, and Hippo, on the occurrence and development of NAFLD-associated hepatocellular carcinoma are also summarized. This analysis will hopefully support the development of diagnostic markers and new therapeutic targets in NAFLD

Recent Advances in Separation and Analysis of Saponins in Natural Products

To better control the quality of saponins, ensure their biological activity and clinical therapeutic effect, and expand the development and application of saponins, this paper systematically and comprehensively reviews the separation and analytical methods of saponins in the past decade. Since 2010, the electronic databases of PubMed, Google Scholar, ISI Web of Science, Science Direct, Wiley, Springer, CNKI (National Knowledge Infrastructure, CNKI), Wanfang Med online, and other databases have been searched systematically. As a result, it is found that ionic liquids and high-performance countercurrent chromatography are the most popular extraction and separation techniques for saponins, and the combined chromatography technique is the most widely used method for the analysis of saponins. Liquid chromatography can be used in combination with different detectors to achieve qualitative or quantitative analysis and quality control of saponin compounds in medicinal materials and their preparations. This paper provides the latest valuable insights and references for the analytical methods and continued development and application of saponins

Galacto-Oligosaccharide Alleviates Alcohol-Induced Liver Injury by Inhibiting Oxidative Stress and Inflammation

Alcoholic liver disease (ALD) is a primary cause of mortality and morbidity worldwide. Oxidative stress and inflammation are important pathogenic factors contributing to ALD. We investigated the protective mechanism of galacto-oligosaccharide (GOS) against ALD through their antioxidant and anti-inflammatory activities by performing in vivo and in vitro experiments. Western blot and RT‒PCR results indicated that the expression of cytochrome P450 protein 2E1 (CYP2E1) in liver tissues and L02 cells was reduced in the GOS-treated mice compared with the model group. In addition, GOS prominently reduced the expression of Kelch-like ECH-associated protein 1 (Keap1), increased the expression of the nuclear factor erythroid-2-related factor 2 (Nrf2) and haem oxygenase-1 (HO-1) proteins, and enhanced the antioxidant capacity. In addition, GOS decreased inflammation by reducing inflammatory factor levels and inhibiting the mitogen-activated protein kinase (MAPK)/nuclear factor kappa B (NF-κB) pathway. Based on these results, GOS may be a prospective functional food for the prevention and treatment of ALD

The Microstructure, Antibacterial and Antitumor Activities of Chitosan Oligosaccharides and Derivatives

Chitosan obtained from abundant marine resources has been proven to have a variety of biological activities. However, due to its poor water solubility, chitosan application is limited, and the degradation products of chitosan oligosaccharides are better than chitosan regarding performance. Chitosan oligosaccharides have two kinds of active groups, amino and hydroxyl groups, which can form a variety of derivatives, and the properties of these derivatives can be further improved. In this review, the key structures of chitosan oligosaccharides and recent studies on chitosan oligosaccharide derivatives, including their synthesis methods, are described. Finally, the antimicrobial and antitumor applications of chitosan oligosaccharides and their derivatives are discussed

Biodegradation and Prospect of Polysaccharide from Crustaceans

Marine crustacean waste has not been fully utilized and is a rich source of chitin. Enzymatic degradation has attracted the wide attention of researchers due to its unique biocatalytic ability to protect the environment. Chitosan (CTS) and its derivative chitosan oligosaccharides (COSs) with various biological activities can be obtained by the enzymatic degradation of chitin. Many studies have shown that chitosan and its derivatives, chitosan oligosaccharides (COSs), have beneficial properties, including lipid-lowering, anti-inflammatory and antitumor activities, and have important application value in the medical treatment field, the food industry and agriculture. In this review, we describe the classification, biochemical characteristics and catalytic mechanisms of the major degrading enzymes: chitinases, chitin deacetylases (CDAs) and chitosanases. We also introduced the technology for enzymatic design and modification and proposed the current problems and development trends of enzymatic degradation of chitin polysaccharides. The discussion on the characteristics and catalytic mechanism of chitosan-degrading enzymes will help to develop new types of hydrolases by various biotechnology methods and promote their application in chitosan

Deep sequencing of HBV pre-S region reveals high heterogeneity of HBV genotypes and associations of word pattern frequencies with HCC

Hepatitis B virus (HBV) infection is a common problem in the world, especially in China. More than 60-80% of hepatocellular carcinoma (HCC) cases can be attributed to HBV infection in high HBV prevalent regions. Although traditional Sanger sequencing has been extensively used to investigate HBV sequences, NGS is becoming more commonly used. Further, it is unknown whether word pattern frequencies of HBV reads by Next Generation Sequencing (NGS) can be used to investigate HBV genotypes and predict HCC status. In this study, we used NGS to sequence the pre-S region of the HBV sequence of 94 HCC patients and 45 chronic HBV (CHB) infected individuals. Word pattern frequencies among the sequence data of all individuals were calculated and compared using the Manhattan distance. The individuals were grouped using principal coordinate analysis (PCoA) and hierarchical clustering. Word pattern frequencies were also used to build prediction models for HCC status using both K-nearest neighbors (KNN) and support vector machine (SVM). We showed the extremely high power of analyzing HBV sequences using word patterns. Our key findings include that the first principal coordinate of the PCoA analysis was highly associated with the fraction of genotype B (or C) sequences and the second principal coordinate was significantly associated with the probability of having HCC. Hierarchical clustering first groups the individuals according to their major genotypes followed by their HCC status. Using cross-validation, high area under the receiver operational characteristic curve (AUC) of around 0.88 for KNN and 0.92 for SVM were obtained. In the independent data set of 46 HCC patients and 31 CHB individuals, a good AUC score of 0.77 was obtained using SVM. It was further shown that 3000 reads for each individual can yield stable prediction results for SVM. Thus, another key finding is that word patterns can be used to predict HCC status with high accuracy. Therefore, our study shows clearly that word pattern frequencies of HBV sequences contain much information about the composition of different HBV genotypes and the HCC status of an individual

Two Novel SNPs in RET Gene Are Associated with Cattle Body Measurement Traits

The rearrangement of the transfection (RET) gene, which mediates the functions of the ganglion in the gastrointestinal tract, plays an important role in the development of the gastrointestinal nervous system. Therefore, the RET gene is a potential factor influencing animal body measurement. The aim of this study was to reveal the significant genetic variations in the bovine RET gene and investigate the relationship between genotypes and body measurement in two Chinese cattle breeds (Qinchuan and Nanyang cattle). In this study, two SNPs (c.1407A>G and c.1425C>G) were detected in the exon 7 of RET gene by sequencing. For the SNP1 and SNP2, the GG genotype was significantly associated with body height, hip height, and chest circumference in Qinchuan cattle (p < 0.05). Individuals with an AG-CC genotype showed the lowest value of all body measurement in both breeds. Our results demonstrate that the polymorphisms in the bovine RET gene were significantly associated with body measurement, which could be used as DNA marker on the marker-assisted selection (MAS) and improve the performance of beef cattle